This repository has been archived by the owner on Jan 22, 2025. It is now read-only.
Ledger cleanup fix #9131
Merged
Ledger cleanup fix #9131
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sakridge
force-pushed
the
fix-ledger-purge
branch
from
2fa48f0 to
da2bdbc
Compare
sakridge
force-pushed
the
fix-ledger-purge
branch
2 times, most recently
from
47b94c8 to
becb91f
Compare
sakridge
commented
Mar 27, 2020
Codecov Report
@@ Coverage Diff @@
## master #9131 +/- ##
======================================
Coverage 81.0% 81.0%
======================================
Files 276 276
Lines 60588 60678 +90
======================================
+ Hits 49078 49171 +93
+ Misses 11510 11507 -3 |
|
@carllin mind taking a look when you get a chance? |
carllin
reviewed
Mar 29, 2020
carllin
reviewed
Mar 29, 2020
carllin
reviewed
Mar 29, 2020
carllin
reviewed
Mar 29, 2020
sakridge
force-pushed
the
fix-ledger-purge
branch
5 times, most recently
from
87a1bbf to
e21da9f
Compare
* Fix purging happening every slot when cleanup service is not started at slot 0 * Purge by shred count instead of slots since slots can have variable number of shreds
sakridge
force-pushed
the
fix-ledger-purge
branch
from
e21da9f to
140d97d
Compare
carllin
reviewed
Mar 31, 2020
carllin
approved these changes
Mar 31, 2020
mergify bot
pushed a commit
that referenced
this pull request
Mar 31, 2020
* Fix purging happening every slot when cleanup service is not started at slot 0 * Purge by shred count instead of slots since slots can have variable number of shreds (cherry picked from commit 73e99cc)
mergify bot
pushed a commit
that referenced
this pull request
Mar 31, 2020
* Fix purging happening every slot when cleanup service is not started at slot 0 * Purge by shred count instead of slots since slots can have variable number of shreds (cherry picked from commit 73e99cc)
solana-grimes
pushed a commit
that referenced
this pull request
Mar 31, 2020
solana-grimes
pushed a commit
that referenced
this pull request
Mar 31, 2020
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Ledger purge logic is broken, it makes the assumption that the node starts from slot 0, which is almost never the case. It also works on slots which is not an accurate measure of the space taken by the ledger since a slot can have a variable amount of data. The first assumption means that when
--no-rocksdb-compactionoption is enabled, a purge is running every slot for at least the first 300,000 slots causing really poor insertion performance.Summary of Changes
Convert to checking if the number slots is greater than some interval before doing the purge. Also, switch the unit of measure to shred.
Fixes the problems with high insertion time seen with #9009 when
--no-rocksdb-compactionoption is enabled.Fixes #